19. (Ne^) The method according to claim 11, wherein the graphic image is a 
virtual image corresponding to a 3-dimensional view of the concrete place. 

Enclosed please find an tc Un-Marked Version of the Claims as Amended" including all of the 
i\ \\ amendments made to claims 1 to 10 above. In addition, the "Un-Marked Version of the Claims as 



Cfl 



Amended" includes new claims 11-19 added herein. It is respectfully submitted that the subject matter 
of new claims 1 1-19 is commensurate in scope with the subject matter of original claims 1-9 and is 
supported by the original specification. 

Please note, additions to the claims are denoted by underlining and deletions from the claims 
are denoted by bracketing. 

REMARKS 

The specification has been amended to include section headings at appropriate locations 
and to correct minor grammatical errors. 

The claims have been amended to remove multiple dependencies therefrom in order to 
reduce the filing fee and to more clearly define the invention. New claims 11-19 have been added 
which are commensurate in scope with original claims 1-19. 

Moreover, the amendments to the claims made herein have been made to conform the 
claims to U.S. practice and have not been made for purposes of patentability. 
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An early and favorable action on the merits is earnestly solicited. 

Respectfully submitted, 



STEINBERG & RASKIN, P C. 




Steinberg & Raskin, P C. 
1 140 Avenue of the Americas 
New York, New York 10036 
Tel.: (212) 768-3800 

Ends. 

- Marked-Up Version of the Substitute Specification 

- Un-Marked Version of the Substitute Specification 

- Un-Marked Version of the Claims as Amended 



Martin G. Raskin 
Reg. No. 25,642 
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Marked-Up Version of the Substitute Specification 989. 1032 J£q 



[A method in a process control system and a process control system] 

-A METHOD IN A PROCESS CONTROL SYSTEM AND 
A PROCESS CONTROL SYSTEM- 



- FIELD OF THE INVENTION - 

1^ The invention relates to a method [according to the preamble of claim 1] in a process 
control system in which a terminal displays symbols illustrating parts of a process and 
10 information about the status of the process . The invention also relates to a process control 
system. 

- BACKGROUND OF THE INVENTION - 

2L In automatic control of industrial processes, real-time data and history data about the 
15 status of the process are displayed to a user at a terminal. For process control systems, there are 
graphic user interfaces, by means of which information about different parts of the process can 
be displayed graphically (with graphs, diagrams, tables, process graphic diagrams consisting of 
symbols). These user interfaces can be based on different operating systems which make it 
possible to process data for example in the form of windows on the screen of a display device. 
20 The different parts of the process, either as large systems or as more detailed images, are 

displayed to the user on the display device of the terminal as displays which may contain process 
charts, graphs, tables and, in general, any information illustrating the parts of the process and the 
run of the process. 

1 



1 In connection with the terminal, [also] an input device is also provided, [to give] which 
input device gives commands for controlling the process. The display device of the terminal is 
normally a display screen in a control room or the like, and the input device is normally a 
keyboard with its peripheral devices, including a device for controlling a cursor or pointer 
movable on the screen (e.g. a mouse). The input device can also be the screen of the display 
device itself, which is capable of directly receiving control commands, such as touches. The 
terminal can also be stationary on the field (outside the control room, near the process), or a 
portable or wearable device. 

4 The displays can be arranged hierarchically so that, for example, by selecting a part of a 
display on a higher level of hierarchy, a display on a lower level of hierarchy is displayed. In a 
larger view on the display device, this part of the display can be displayed as a certain framing or 
in another way, and this can be selected by means of the cursor or pointer. Furthermore, the 
displays can be linked in such a way that by selecting a link in a display, a display linked with 
this display is accessed. Furthermore, it is possible that displays can be selected from menus 
which can be retrieved in the display device, or the address of the image can, in principle, be also 
directly written, if the displays are identified with specific address data, such as file names. 

y Graphic user interfaces applying data in the form of various displays to control and 
monitor an industrial process are presented e.g. in US patents 5,918,233 and 5,576,946 as well as 
in European patent 72161 1 . It is typical of such user interfaces that the operator (person 
monitoring the process) can control the process by means of the user interface, i.e. to enter data 
by the input device to control the process and to monitor the run of the process, observe 




disturbance messages or alarms from different locations, etc. Furthermore, as is typical for the 
programs, situation specific instructions related e.g. to the relevant process portion or process 
step can be shown in the user interface. 

5 *L A problem in the above systems is that for the purpose of process control, a process 
graphic diagram displayed on the display device of a terminal does not give a good picture of, 
for example, the physical location of various devices (pumps, valves, measuring devices, etc.) in 
the plant in which the process takes place. The chart displayed on the display device is a process 
graphic diagram based on the operation of the process and the devices related thereto. If the 
10 operating personnel detects, on the display, a disturbance message concerning, for example, a 
pump, this particular pump cannot be physically located on the basis of the process graphic 
diagram. 

1 Furthermore, US patents 5,880,716 and 5,982,362 disclose systems in which a video 
1 5 camera is placed in the field. By means of this video camera, a real image is produced of the 
object, for example simultaneously with the corresponding process graphic diagram. The 
purpose of the systems is to facilitate the control of the process, and by means of them, it is 
possible, for example, to monitor in real time how the process is affected by the control 
measures taken at the terminal. For example, US patent 5,412,400 shows a display device in a 
20 process monitoring system where a real image taken by a television camera can be displayed in a 
separate window on a screen. 

8^ In addition, in process control technology, an aim has been to improve the informativity 

3 




of real images. Therefore, for example German application publication 1954065 discloses a 
method for providing a stereoscopic effect in an image formed on a projection surface. 

9. It is true that by the video technology disclosed in the above-mentioned publications it is 
5 possible to monitor real objects corresponding to symbols shown in process graphic diagrams, 
for example their behaviour and the effect of control measures on them, but this will hardly 
facilitate finding them in the field. 

— OBJECTS AND SUMMARY OF THE INVENTION — 

10 KX It is an aim of the invention to eliminate the aforementioned drawbacks and to present a 
method which is more capable than those of the prior art of giving information about the process 
both in process graphic diagram format and in the form of images illustrating the process 
environment itself (the field). [To attain this purpose, the method according to the invention is 
primarily characterized in what will be presented in the characterizing part of the appended 

15 claim 1.] 

11. To the portions of the process graphic diagram displayed on the terminal are allotted 
graphic images indicating the physical location of the respective objects and stored as separate 
files. The graphic image can be displayed by the control device of the terminal, for example by 
20 activating the corresponding portion of the chart. This graphic image can be a three-dimensional 
image which is preferably of a type whose scale can be changed and which can be turned in 
different angles of viewing, and preferably it is possible to "wander" in it so that, after enlarging, 
the image can be turned again, that is, it is possible to "dive" into the image. The graphic image 
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is not a video image but an artificially produced picture, best described as a "virtual image", 
which corresponds to a real view of the plant and which preferably can be controlled by a 
graphics program in the user interface. The graphic image can also be a two-dimensional map- 
like representation, for example a ground plan indicating the location of the object. The graphic 
image gives a good impression on where, for example, a place corresponding to a given portion 
in the process chart is located in the field itself, that is, it gives, in a way, the location of the 
place in a three-dimensional coordinate system. This considerably increases the possibilities for 
using the system for example in disturbance situations, in which one must, for example, get to 
the target location in the field. By means of the invention, malfunctioning objects can be located 
faster, and the disturbances can be removed faster. The invention can also be used for training, 
wherein for example new operating personnel becomes quickly familiarized with the plant, in 
which the process takes place. 

12. In particular, the invention is suitable for use in connection with such a terminal which is 
movable, that is, it comprises a movable display device and an input device which can be carried 
along when moving in the plant. It can be a portable terminal or a so-called wearable terminal, in 
which latter case the display can be fixed for example at the height of the user's head in such a 
way that it can be monitored when walking in the plant. 

13. In this context, three-dimensional does not mean an image which should necessarily have 
a stereoscopic effect, but it is an image which graphically illustrates a three-dimensional space 
and from which the locations of different elements can be determined. By turning the image into 
different positions, a particularly informative three-dimensional image is obtained, because it 



can display, for example, parts concealed behind some elements, and it is thus more informative 
than a mere single standard, possibly zoomable view which is seen from one direction and which 
is by some means provided with a stereoscopic effect, or a mere video camera image. 

14. Also other information can be connected with the graphic image, for example written 
location instructions, "navigation instructions". It is also possible that a 3-dimensional view and 
a ground plan image can be displayed together, for example in separate windows. 

15. Further, it is possible to arrange an automatic function in connection with the graphic 
image in such a way that it displays an overall view of the target, for example a whole plant in 
which the process takes place, and after this, it zooms in on the target to be found, thus 
displaying it in a larger scale. This function can be arranged both in a 3-dimensional view and in 
a ground plan. 

- BRIEF DESCRIPTION OF THE DRAWINGS - 

16. In the following, the invention will be described in more detail with reference to the 
appended drawings, in which 

Fig. 1 shows the operating environment of a control system as a diagram, and 

Fig. 2 shows one feasible embodiment of the invention as an image displayed on 

a display device in the user interface. 



- DETAILED DESCRIPTION OF THE INVENTION - 

17. Figure 1 shows the operating environment of a process control system. The process 1 is, 



in principle, any industrial process comprising several variables which can be measured and 
which are monitored, controlled and/or adjusted. Thus, the process can comprise, for example, 
one or more closed control loops. Level 2 "Measurements, adjustments, controls of the control 
system" describes the measurements by the system and the operations conducted by the system. 
The arrow from the process describes information obtained from the process, such as measured 
values and/or status data. The arrow to the process describes measures to control and/or adjust 
the process. The third level describes the user interface software 3, by means of which the 
operator can monitor and control the process. The user interface software 3 is in bidirectional 
communication with the measurements, adjustments and controls of the control system. The user 
interface software 3 comprises several graphic images 5 which can be viewed with a terminal 4 
using the user interface software. Images 5 show several objects which correspond to a specific 
location that can be defined with 3 -dimensional coordinates in a plant in which the process to be 
controlled takes place. The terminal comprises a display device 4a and an input device 4b. The 
input device can comprise, for example, a keyboard and a mouse to move a cursor movable on 
the display and to control the functions produced by it. The input device 4b can also be a display 
screen operated by pointing with a finger. The terminal 4 can also be carried along in the field. 

18. To these objects, which are often represented by symbols in the chart, correspond graphic 
images 6 which are indicated with the note "graphic images" in Fig. 1 . These graphic images 
illustrate a concrete object (pump, valve, any other actuator, measuring device or a part of the 
process equipment, such as a tank, a pipe, etc.), generally any concrete part which is displayed as 
a specific symbol or set of symbols in the user interface. This object can be shown in the virtual 
image in such a manner that it is clearly visible, for example it can be represented in a manner 




that distinguishes it from the background of the virtual image. Also the whole image 5 may 
correspond to a certain physical object. The physical objects are shown as virtual images which 
are not camera views of the objects. 

5 1£. Figure 2 shows, as an example, how it is possible to display on the terminal, for example 
by selecting a given part of the process graphic diagram with the input device, a three- 
dimensional graphic image (virtual image) which can be displayed as a separate window or be 
changed over the original graphic diagram or be changed into a window that is already open but 
shows another graphic image. The graphic image window comprises control buttons, menus or 
10 the like, by which it is possible to control the graphic image itself, for example to enlarge it, to 
turn it in different angles of viewing, or to wander inside the graphic image. 

20. When one wants to view a portion of an image that is currently displayed on the display 
device as said three-dimensional graphic image, it can be selected in several different ways. The 

1 5 point of the image may comprise a corresponding button which can be activated to access the 
graphic image. It is also possible that by moving the cursor or pointer in the image, those points 
at which a three-dimensional graphic image is accessible, are displayed and indicated in a 
suitable way. Similarly, by clicking a point in the image, a menu can be displayed, from which 
the graphic image is selected. Furthermore, it is possible that in connection with a certain event 

20 in the process, for example a disturbance, the corresponding three-dimensional graphic image is 
automatically displayed. 

21. The way of displaying the graphic image itself can also vary. It is possible that in the 

8 




same process graphic diagram, particularly when illustrating an overall view, different objects 
are indicated, which can be activated by means of a control device, for example a cursor or 
pointer, to display their details in a larger scale. These details can, in turn, be processed as 
separate graphic images. Such details can be distinguished in the three-dimensional graphic 
5 image for example by circling, by changed background, by a symbol, or in a corresponding way. 
The object which was initially selected in the process graphic diagram can be marked or 
highlighted in the graphic image in some suitable way to find it more easily. 

22. It is also possible that a general view and a partial view are displayed in parallel in the 
10 display device, as separate three-dimensional graphic images which can be separately processed 

(turned, enlarged etc.). The graphic image can also display other information about the object it 
illustrates, for example in the case of a measuring device, the values given by the measuring 
device, or in the case of a tank, for example the level of the tank, or in general, all possible status 
data about the object. 

15 

23. Ground plans can be analogically used in the same way as the 3-dimensional graphic 
images (virtual images), that is, they can be displayed in corresponding windows. 

24. The invention is not restricted solely to the embodiment described above, but it can be 
20 modified within the scope of the inventive idea presented by the claims. 
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[Abstract] 

- ABSTRACT - 

In a process control system, a terminal (4) displays symbols illustrating parts of a process and 
5 information about the status of the process, related to concrete places in the operating 

environment of the process, such as actuators, pumps, measuring devices, process equipment 
parts, or the like. Corresponding graphic images (6) are allotted to said concrete places, 
indicating the location of the place in the process, and these images can be displayed in the 
operating environment of the process control system when the process is running. The image can 
10 be processed when it is displayed, for example in such a way that a 3-dimensional image can be 
turned in different angles of viewing and/or it can be enlarged. 



